Assessing Bias in Face Image Quality Assessment

Face image quality assessment (FIQA) attempts to improve face recognition (FR) performance by providing additional information about sample quality. Because FIQA methods attempt to estimate the utility of a sample for face recognition, it is reasonable to assume that these methods are heavily influenced by the underlying face recognition system. Although modern face recognition systems are known to perform well, several studies have found that such systems often exhibit problems with demographic bias. It is therefore likely that such problems are also present with FIQA techniques. To investigate the demographic biases associated with FIQA approaches, this paper presents a comprehensive study involving a variety of quality assessment methods (general-purpose image quality assessment, supervised face quality assessment, and unsupervised face quality assessment methods) and three diverse state-of-theart FR models. Our analysis on the Balanced Faces in the Wild (BFW) dataset shows that all techniques considered are affected more by variations in race than sex. While the general-purpose image quality assessment methods appear to be less biased with respect to the two demographic factors considered, the supervised and unsupervised face image quality assessment methods both show strong bias with a tendency to favor white individuals (of either sex). In addition, we found that methods that are less racially biased perform worse overall. This suggests that the observed bias in FIQA methods is to a significant extent related to the underlying face recognition system.Comment: The content of this paper was published in EUSIPCO 202

Resource constrained scheduling - a case study using Primavera Project Planner Version 3.1

In the spring semester of the academic year 2005/06 I took part in the Socrates Erasmus student exchange, writing my graduation thesis at the Project Management Centre at Istanbul Technical University. In this thesis I will present the process of planning along with its basic properties and methods, which have to be used when planning a project in the construction sector, using the expanded Critical Path Method. I included some theoretical background of resource scheduling and its possibility of using it with modern software tools, which is presented on a real project prepared by using Primavera Project Planner Version 3.1, a modern programme for planning and managing projects. The techniques I dealt with are levelling, allocation, cumulation, aggregation and smoothing. In order to control the project, the Earned value analysis is presented as well. Apart from all the aforementioned methods and techniques, I also decided to portray how the professionals in construction should use these techniques with modern software tools for managing and planning the project during its life, including the reports that are made right after a particular plan is finished, as well as other reports compiled during the project, if the project contains all necessary data. These reports are: schedule, resource/cost control, resource/cost loading, productivity, cash flow, earned value, tabular resource/cost and matrix reports made with P3. In the end, the solutions for sending and sharing project data via the web or local servers between the members of a project team, as well as the responsibilities of the latter for resource scheduling and managing the project using P3 are depicted

Optimization-Based Improvement of Face Image Quality Assessment Techniques

Contemporary face recognition (FR) models achieve near-ideal recognition performance in constrained settings, yet do not fully translate the performance to unconstrained (realworld) scenarios. To help improve the performance and stability of FR systems in such unconstrained settings, face image quality assessment (FIQA) techniques try to infer sample-quality information from the input face images that can aid with the recognition process. While existing FIQA techniques are able to efficiently capture the differences between high and low quality images, they typically cannot fully distinguish between images of similar quality, leading to lower performance in many scenarios. To address this issue, we present in this paper a supervised quality-label optimization approach, aimed at improving the performance of existing FIQA techniques. The developed optimization procedure infuses additional information (computed with a selected FR model) into the initial quality scores generated with a given FIQA technique to produce better estimates of the "actual" image quality. We evaluate the proposed approach in comprehensive experiments with six state-of-the-art FIQA approaches (CR-FIQA, FaceQAN, SER-FIQ, PCNet, MagFace, SDD-FIQA) on five commonly used benchmarks (LFW, CFPFP, CPLFW, CALFW, XQLFW) using three targeted FR models (ArcFace, ElasticFace, CurricularFace) with highly encouraging results.Comment: In proceedings of the International Workshop on Biometrics and Forensics (IWBF) 202

An overview of employment contract and social security breaches in the Repulic of Slovenia

Diplomsko delo z naslovom Pregled kršitev delovnega razmerja in socialne varnosti v Republiki Sloveniji obravnava vrste in pogostost posameznih kršitev, ki nastajajo zaradi neizpolnjevanja obveznosti iz delovnega razmerja in predstavljajo kršenje delavčevih pravic. Skozi nalogo smo z analiziranjem statističnih podatkov o zaznanih kršitvah delovnega razmerja in socialne varnosti med leti 2004 in 2012 primerjali število kršitev pred in po pojavu finančne krize leta 2008. Predmet analize so bili statistični podatki Policije, Finančne uprave Republike Slovenije in Inšpektorata Republike Slovenije za delo, s pomočjo katerih smo ugotovili, da je finančna kriza imela močan vpliv na rast števila kršitev, predvsem na področju tistih, ki so neposredno povezane z finančno stabilnostjo podjetij. Največja rast je bila ugotovljena na področju kršitev temeljnih pravic delavcev, kjer prevladuje delodajalčevo neizpolnjevanje obveznosti pri plačevanju za delavčevo opravljeno delo in pri plačevanju obveznih prispevkov za socialno varnost, kjer je dolg neplačnikov do države, ugotovljen s strani Finančne uprave RS, vsako leto večji. Pri analizi dela nadzornih organov smo ugotovili, da se glavni razlogi za neustrezno spopadanje s porastom števila kršitev skrivajo v neustrezni zakonodaji, kjer velja izpostaviti neustrezno sankcioniranje prekrškov, zaradi težavne dokazljivosti naklepa kaznivega dejanja malo število pravnomočno obsojenih kršiteljev in enostavno ustanavljanje novih podjetij bivših kršiteljev. Poleg zakonodaje pa delo nadzornih organov, zaradi omejitev zaposlovanja v javnem sektorju, otežuje kadrovska stiska. Za izboljšanje stanja na področju boja proti kršitvam delovnega razmerja in socialne varnosti je tako največ rezerve prav v ustreznih dopolnitvah zakonodaje s strani države.The title of this dissertation is An overview of employment contract and social security breaches in the Republic of Slovenia. It addresses different types and the frequency of individual violations, which occur due to failures to comply with obligations arising from the employment contract and constitute a violation of employee\u27s rights. Throughout this dissertation we have analysed statistical data of detected violations of employment contract and social security between the years 2004 and 2012, and compared the number of violations before and after the beginning of the financial crisis in 2008. The subject of analysis was statistic data collected from the Police, Financial administration of the Republic of Slovenia, and Labour Inspectorate of the Republic of Slovenia. The data helped us reach the conclusion that the financial crisis had a great impact on the growth of the number of violations, most of which were directly connected to financial stability of different companies. The biggest growth was documented in the area of violations of fundamental workers’ rights, such as the employer’s deficiency to pay for the work performed by the employee, and the lack of paying for mandatory social security contributions. The non-payers’ debt to the country is bigger each year, claims the Financial administration of the Republic of Slovenia. By analysing the work of statutory authority we established that the main reasons for inadequate coping with the increasing number of violations hide behind the insufficient legislation. We have to highlight the inadequate sanctioning of offences, due which there is a small number of convicted offenders, because of difficulties in proving the intent of a criminal act, and a trouble-free setting up of new businesses of former offenders. Apart from the legislation, the work of statutory authority is, because of limitations of employment in the public sector, made difficult by the shortage of personnel. In the fight against the violations of employment contract and social security the state still has the most reserve in the apt amendment of the legislation to improve this situation

Song similarity analysis based on lyrics, ratings and meta data

Robustnost in natančnost glasbenega priporočilnega sistema je povezana s kvaliteto in tipom podatkov, ki jih ta upošteva. Različne vrste podatkov se razlikujejo po zahtevnosti analize in pridobivanja. Podatke, ki jih je težje pridobiti in analizirati, želimo nadomestiti s podatki, ki nosijo enako informacijo in jih je lažje pridobiti in analizirati. Analiziramo podatkovne nabore besedil pesmi, ocen pesmi in glasbenih meta podatkov. Iz podatkovnih naborov zgradimo matrike podobnosti pesmi s pomočjo tekstovnega rudarjenja, analize omrežij in analize vektorjev. Matrike primerjamo z različnimi merami in rezultate primerjanj zapišemo v matrike podobnosti naborov. Pregled matrik podobnosti naborov omogoča odkrivanje skritih podobnosti med različnimi glasbenimi podatki. Izkaže se, da je podobnost med nabori omejena na tip podatkov in način analize podatkov.The accuracy and robustness of song recommendation systems depends on the quality and type of data given to the system. Different types of data vary in difficulty of extraction and analysis. We wish to replace data which is harder to extract and analyse, with data that carries the same information, yet is easier to extract and analyse. The extracted data sets include song lyrics, song popularity scores and song meta data. From the data sets we build song similarity matrices for each data set, using text mining, network analysis and vector analysis. Song similarity matrices are then compared using five different measures, and the results are stored in data set similarity matrices. A thorough examination of data set similarity matrices can reveal hidden similarities between different data sets. Results show that similarity between different data sets is limited to the type of data and type of analysis

Face Deidentification using Face Swapping

V zadnjih letih količina vizualnih vsebin, ki vsebujejo občutljive informacije o identiteti posameznikov, drastično raste. Posledično se razvijajo in širijo učinkoviti mehanizmi za zagotavljanje zasebnosti posameznikov. V delu se osredotočamo na razvoj takšnih mehanizmov za obrazne slike, ki jih lahko uporabimo v sistemih za samodejno razpoznavanje obrazov. Med obstoječimi rešitvami se v literaturi pogosto pojavijo postopki deidentifikacije, ki iz podatkov odstranijo informacijo o identiteti in ohranijo manj občutljive informacije. Za deidentifikacijo slik uporabimo različne mehanizme za zagotavljanje zasebnosti obraznih slik. Naš glavni cilj je preveriti zmožnost delovanja takšnih mehanizmov v kombinaciji s pristopom za zamenjavo obrazov. V delu se omejimo na tri mehanizme za zagotavljanje zasebnosti obraznih slik. Prvi mehanizem temelji na dodajanju nasprotniških perturbacij v obrazno sliko. Za izvedbo mehanizma smo izbrali metodo hitrega predznačenega gradienta, ki jo nadgradimo z uporabo ansambla modelov ter binarnimi obraznimi maskami. Drugi mehanizem temelji na shemi k-enakosti, kjer s pomočjo rojenja in generatorja StyleGAN ustvarimo umetne obrazne identitete, s katerimi zamenjamo obraze v izvirnih slikah. Zadnji mehanizem temelji na ε-diferencialni zasebnosti, kjer ustvarjamo umetne obrazne identitete z vnašanjem šuma v StyleGAN-vložitve slik. Rezultati mehanizmov za zagotavljanje zasebnosti kažejo, da so vse implementacije sposobne deidentificirati obrazne slike do neke mere. Z vključevanjem mehanizmov v pristop za zamenjavo obrazov pa se sposobnost deidentifikacije le-teh zmanjša. Združevanje generativnih mehanizmov, kot sta shema k-enakosti ter ε-diferencialna zasebnost, z nasprotniškimi mehanizmi v splošnem izboljša rezultate v primerjavi z uporabo posameznih mehanizmov tudi, ko jih združimo s pristopom za zamenjavo obrazov.The amount of visual content containing sensitive identity information has steadily grown in the past years. This has led to the development of effective privacy enhancing mechanisms. In our work we focus on face images, which can be used in automatic face recognition systems. The approaches presented in related papers frequently use de-identification mechanisms, which remove only identity information from the data, while preserving other information. For the purpose of de-identification of face images we use different privacy enhancing mechanisms. Our main goal is to verify if the mechanisms can be used with a face swapping approach. In our work we focus on three different privacy enhancing mechanisms. The first is an adversarial approach, which adds perturbations into the face image. We chose to use the Fast Gradient Sign Method, which we enhanced using an ensemble of models and binary face masks. The second mechanism is based on the k-same method, where we generate artificial face identities using clustering and the StyleGAN generator, which replace the faces of the source images. The last mechanism is based on ε-differential privacy, where we generate artificial face identities by introducing noise to the StyleGAN embeddings of images. The results of individual privacy enhancing mechanisms showed that all implementations were able to provide some form of de-identification. Combining individual mechanisms with a face swapping approach lowered the de-identification capabilities of all mechanisms. Using prior mechanisms such as the k-same method or ε-differential privacy with posterior adversarial mechanisms seemed to further improve de-identification capabilities, even when combined with a face swapping approach

DifFIQA: Face Image Quality Assessment Using Denoising Diffusion Probabilistic Models

Modern face recognition (FR) models excel in constrained scenarios, but often suffer from decreased performance when deployed in unconstrained (real-world) environments due to uncertainties surrounding the quality of the captured facial data. Face image quality assessment (FIQA) techniques aim to mitigate these performance degradations by providing FR models with sample-quality predictions that can be used to reject low-quality samples and reduce false match errors. However, despite steady improvements, ensuring reliable quality estimates across facial images with diverse characteristics remains challenging. In this paper, we present a powerful new FIQA approach, named DifFIQA, which relies on denoising diffusion probabilistic models (DDPM) and ensures highly competitive results. The main idea behind the approach is to utilize the forward and backward processes of DDPMs to perturb facial images and quantify the impact of these perturbations on the corresponding image embeddings for quality prediction. Because the diffusion-based perturbations are computationally expensive, we also distill the knowledge encoded in DifFIQA into a regression-based quality predictor, called DifFIQA(R), that balances performance and execution time. We evaluate both models in comprehensive experiments on 7 datasets, with 4 target FR models and against 10 state-of-the-art FIQA techniques with highly encouraging results. The source code will be made publicly available

Optimization-Based Improvement of Face Image Quality Assessment Techniques

Contemporary face recognition (FR) models achieve near-ideal recognition performance in constrained settings, yet do not fully translate the performance to unconstrained (real-world) scenarios. To help improve the performance and stability of FR systems in such unconstrained settings, face image quality assessment (FIQA) techniques try to infer sample-quality information from the input face images that can aid with the recognition process. While existing FIQA techniques are able to efficiently capture the differences between high and low quality images, they typically cannot fully distinguish between images of similar quality, leading to lower performance in many scenarios. To address this issue, we present in this paper a supervised quality-label optimization approach, aimed at improving the performance of existing FIQA techniques. The developed optimization procedure infuses additional information (computed with a selected FR model) into the initial quality scores generated with a given FIQA technique to produce better estimates of the "actual" image quality. We evaluate the proposed approach in comprehensive experiments with six state-of-the-art FIQA approaches (CR-FIQA, FaceQAN, SER-FIQ, PCNet, MagFace, SER-FIQ) on five commonly used benchmarks (LFW, CFP-FP, CPLFW, CALFW, XQLFW) using three targeted FR models (ArcFace, ElasticFace, CurricularFace) with highly encouraging results